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1^ ■ Abstract 

! The asymptotic iterative decoding performances of low-density parity-check (LDPC) codes 

I using min-sum (MS) and sum-product (SP) decoding algorithms on memoryless binary-input 

output-symmetric (MBIOS) channels are analyzed in this paper. For MS decoding, the analysis 

^ . is done by upper bounding the bit error probability of the root bit of a tree code by the 

^ [ sequence error probability of a subcode of the tree code assuming the transmission of the all- 

, zero codeword. The result is a recursive upper bound on the bit error probability after each 

iteration. For SP decoding, we derive a recursively determined lower bound on the bit error 

I probability after each iteration. This recursive lower bound recovers the density evolution 

' equation of LDPC codes on the binary erasure channel (BEC) with inequalities satisfied with 

■ equalities. A significant implication of this result is that the performance of LDPC codes under 

I SP decoding on the BEC is an upper bound of the performance on all MBIOS channels with 

' the same uncoded bit error probability. All results hold for the more general multi-edge type 

^ ' LDPC codes. 

O ; 

'. 1 Introduction 

^ Low-density parity-check (LDPC) codes, first introduced by Gallager [1], have been shown to be 
powerful channel codes under iterative decoding by numerous simulations in the literature. How- 
ever, the loopy graphical structure of the LDPC codes generally prohibits their iterative decoding 
performance from being exactly analyzed. This problem has been partially solved in [2] by consider- 
ing the asymptotic average performance of an ensemble of LDPC codes when their codeword length 
goes to infinity. In particular, the authors in [2] proved that the performance of each code in the 
ensemble asymptotically approaches the average performance of the ensemble with loop-free local 
structure within finite iterations as the codeword length goes to infinity. This result gave birth to 
the density evolution (DE) method proposed in [2], and was used to provide the exact asymptotic 
performance of LDPC codes after an arbitrary number of iterations on memoryless binary-input 
output-symmetric (MBIOS) channels. 

A major drawback of the DE method is that the evolved densities in general require an infinite 
dimensional description. Therefore, DE is not suitable for the derivation of analytical performance 
bounds. To solve this problem, several approaches have been proposed that track the evolution 
of the densities projected to some specific finite dimensional space. These include the Gaussian 



approximation [3], the extrinsic information transfer (EXIT) chart [4], and the generahzed EXIT 
(GEXIT) [5] chart methods. Unfortunately, since the EXIT and GEXIT chart methods still re- 
quire numerical calculations, and the Gaussian approximation does not imply any upper or lower 
bounds on the exact performance, these results still can not be used to analytically bound the code 
performance. 

In [6], the authors propose to map the evolved densities to Bhattacharryya parameters, which 
are further used to bound the bit error probability of the LDPC codes under sum-product (SP) 
decoding [7] on MBIOS channels. It turns out that an upper bound on the Bhattacharryya pa- 
rameters can be obtained by a recursion involving only one-dimensional real numbers, so the whole 
result can be used to determine a guaranteed decoding capability for the LDPC codes. In this 
paper, we improve this result of [6] by showing that the same recursively determined upper bound 
on the bit error probability of the LDPC codes not only holds for the SP decoding, but also holds 
for the min-sum (MS) decoding [7]. This result is attained by upper bounding the probability of 
error of the root bit of a tree code by a sequence error probability of a subcode of the tree code, 
and then using the union bound. Therefore, the whole proof does not involve the Bhattacharryya 
parameters. 

Then, we turn our attention to SP decoding and derive a recursive lower bound on the proba- 
bility of bit error after each iteration. This recursive lower bound becomes exact and recovers the 
one-dimensional density evolution equation on the binary erasure channel (BEG). Exploiting the 
resemblance of our recursion to the DE equation on the BEG, we prove that LDPG codes under 
SP decoding on an MBIOS channel with uncoded bit error probability Pq always have an equal or 
worse performance than that on the BEG with erasure probability e = 2Po- This result should be 
compared with the result in [8], where a tighter recursive upper bound than that in [6] on the Bhat- 
tacharryya parameters associated with the outgoing message of bits (and hence an upper bound 
on the bit error probability) after each iteration is derived, which also exactly recovers the density 
evolution equation on the BEG. Therefore, the iterative decoding analysis of LDPG codes on the 
BEG can be used to bound the performance of LDPG codes from below and above on all MBIOS 
channels via our result and the result in [8]. Note also that, due the nature of the proofs of the 
main lemmas, this connection between BEG and MBIOS channels is also true for the general family 
of multi-edge type LDPG codes [9], including the irregular repeat-accumulate (IRA) codes [10,11] 
and the low-density parity-check and generator matrix (LDPG-GM) codes [12]. 

The remaining of this paper is structured as follows. In Section [21 we review the preliminary 
background on MBIOS channels and the asymptotic analysis of LDPG codes. Then, we present our 
asymptotic performance analysis of LDPG codes on MBIOS channels under MS and SP decoding 
in Section El and El respectively. Finally, we conclude this paper in Section 

2 Preliminaries 

If X G {0, 1} and y G M are the input and output symbols, respectively, of an MBIOS channel with 
conditional density function f{y\x), then we have the following symmetry condition: 

f{y\0) = f{-y\l), VyGM (1) 

Note that all our results hold also when y is a discrete random variable, in which case, we simply 
treat f{y\x) as a conditional probability mass function and integration signs as summation signs 
wherever necessary. However, for convenience, we will assume y to be a continuous random variable 
throughout this paper. To analyze the asjTiiptotic average iterative decoding performance of a (A, p) 



irregular LDPC ensemble when the codeword length goes to infinity, where \{x) = AjX* ^, 

and p{x) = Pi^^~^ the standard variable and check node degree distributions, respectively, 
from the edge perspective as defined in [13], it is shown in [2] that we can as well consider the 
cycle-free case. In this case, the probability of bit error after / decoding iterations is the probability 
of decoding error of the root bit on the tree of / + 1 (variable node) levels whose construction is 
dictated by the degree distributions (A,p) as in [13]. Due to the symmetry condition of MBIOS 
channels, we can also assume without loss of generality that the all-zero codeword is transmitted. 
Hence, in the following two sections, we will analyze the asymptotic MS and SP iterative decoding 
performance of the (A, p) LDPC ensemble on MBIOS channels by considering the corresponding 
tree codes and assuming the transmission of the all- zero codeword. 



3 Min-Sum Decoding Performance Analysis 

Consider an arbitrary binary tree code whose codebook is C = {cq = 0,Ci, . . . ,Cjv/}, where Cj = 
(qi, Ci2, • • • , Cin) is a codeword of length n for all i, and Cq is the all- zero codeword. Let qi be the 
root bit of this tree for all i, x = (xi, X2, . . . , Xn) the transmitted codeword, and y = (yi, ?/2, • • • , Vn) 
the received sequence from an MBIOS channel with conditional probability mass function p{y\x). 
When min-sum (MS) decoding is performed on this tree, it essentially performs maximum likelihood 
sequence detection (MLSqD) on the whole sequence to produce an estimate c = (ci, C2, . . . , c^). 
Therefore, if we define the decision region for the codeword as 

U,^{^U,k: (2) 

where 

{n n ^ 

y e M"| X[p{yj\c,j) > l[piy,\ckj) \ , (3) 

then the probability of the root bit being in error under MS decoding assuming x = is 

pMS = p^(g^ = = Co) = Pr I y G 11 W,|x = Cq 1 . (4) 



Prlye [j 



We would like to find a compact representation of the set 

W= \J U,: (5) 

i,Cii=l 

or at least a superset of U so that we can bound P^'^^ from above. Note that such an upper bound is 
also a valid upper bound on the probability of root bit error P^^ when the SP decoding is performed 
on the tree since the SP decoding essentially performs optimal maximum likelihood decoding for 
each bit. 



Definition 1 Define the subcode of C as the set of codewords in C such that 



(i) the root hit is 1, and 



(a) each check node with parent variable node equal to 1 has exactly one child variable node equal 
to 1, and 



(Hi) each check node with parent variable node equal to has all children variable nodes equal to 
0. 

We show how we can use this reduced codebook Cr to characterize U in the following lemma. 

Lemma 1 U C U,c,GC.^io- 

Proof: Given any y E U = IJicii=i^«' ^^ere exists a k such that y E lAk and Cki = 1. To 
proceed with our proof, we first carry out the following labeling procedure on the codeword Cfc. 

1. At the initial state, the root bit is labelled as "survivor", and all the other variable nodes are 
unlabelled. Note that in this labeling procedure, the "survivor" variable nodes will always 
have value 1 in c^. We first consider the check nodes at the topmost level of the tree. 

2. For every check node c at this level whose parent node is labelled as "survivor", it must have 
at least one child variable node with value 1 in since "survivor" nodes always have value 
1 in Cfc. Choose an arbitrary child variable node of c with value 1 in c^, and label it as 
"survivor" . Then, label the subtrees emanating from the other unlabelled child variable nodes 
of c as "dropped" . 

3. If there are no check nodes at the next lower level of the tree, stop. Otherwise, move to the 
check nodes at the next lower level and go back to 2. 

As we can see after this labeling procedure, the check nodes with a "survivor" parent node all 
have exactly one "survivor" child node, and the ones with a "dropped" parent node have purely 
"dropped" child nodes. Therefore, if we let 

for all "dropped" bits , , 

(6) 

1 = Cfc for all "survivor" bits 

then we have Cm E Cr CI C. In the following, we would like to prove that y G Umo, which completes 
the proof. Let 

{Cfc for all "dropped" bits , , 

(7) 
for all "survivor" bits 

i.e., let C; be the bitwise XOR of and c^. Then, since and are valid codewords in C, so is 
C;. Moreover, since y Ehlk implies y G Um, we have from Q that 

n n 

j=l j=l all "survivor" bits j all "survivor" bits j 

n pivjKj) > n 

all "survivor" bits j all "survivor" bits j 

n n 

^\{p{y^Cm,)>\{p{y^Q) (8) 



which proves that y G Umo as desired. 



This lemma shows that we can upper bound by the probabihty of MLSqD error p^^^^i'^ on 
the reduced codebook Cr assuming that the all-zero codeword is transmitted. One way to proceed 
from here is to use union bound and the fact (see [14, Theorem 7.5] for a proof) that 

Pr(y gW.o|x = Co) <D'^('^'\ Vz (9) 

where w{ci) denotes the Hamming weight of Cj, and D is the Bhattacharryya parameter associated 
with the MBIOS channel p{y\x), to further upper bound p^^^^i^^ For this purpose, we would like 
to introduce the weight enumerator Ni{x) of the reduced codebook Ci of the tree code Gi of level 
/ + 1 associated with a randomly drawn code from the (A, p) LDPC ensemble. Let Ai be the number 
of codewords of weight i in Q. We define Ni{x) by Ni{x) = Aix\ Moreover, let Ni{x) denote 
the expected value of Ni{x) averaged over the whole (A,p) LDPC ensemble. We have the following 
lemma. 



Lemma 2 Nnix) = x and 



Ni{x) = A(p'(l)iV,„i(x)), V/ > 1 (10) 
where p'{x) denotes the derivative of p{x). 



Proof: It is obvious that A'o(a;) = x. To prove the recursion for Ni(x), first consider the subtree 
emanating from the zth check node Cj immediately below the root bit. Let Zi^\x) denote the weight 
enumerator of the reduced codebook of this subtree. Since, the root bit is 1 for all codewords in 
the reduced codebook Ci, there is exactly one child subtree with root 1 emanating from q for all 
codewords in Q. Therefore, if Cj has degree dc, then we have 



Z«(x) = (4 - l)iV,„i(x) Z«(x) = ^(2 - l)iV,„i(x)p, = p'(l)iV,„i(x). (11) 



i=l 



Similarly, if the root bit has degree dy, then we have 



oo dc — 1 oo dc — 1 



Ni{^) = n ^^^^(^) ^ ^'(^) = E n ^?^^(^)^^- = E n ^/^^(^)^^- = ^(^)) (12) 

1=1 j=l i=l i=j 1=1 

where the second equality follows from the fact that the subtrees emanating from different q's are 

generated independently, and the third equality follows from the fact that zj^^"^ (x) does not depend 
on i as shown in (fTT|) . Combining (fTTj) and ()12|) . the lemma is proved. ■ 
Using this lemma and the union bound on p^^^'^i^^ have the following theorem. 

Theorem 2 Given any (A, p) LDPG ensemble, let P^^^ and Pf^ be its asymptotic (as the codeword 
length approaches infinity) average bit error probability after I iterations under MS and SP decoding, 
respectively, on an MBIOS channel with Bhattacharryya parameter D. If we define the sequence 
{zi}^o h zo = D, and Zi = X{p'{l)zi.i), for alll>l, then we have ^ < P^*'^^ < Zi, for all I > 0. 



Proof: From Lemma |21 we see that Zi = Ni{D) for all /. Now, the lemma follows from the 
union bound on p^^^^i^ as discussed above. ■ 
A similar result to Theorem |21 is established in [6, Lemma 1]. However, Theorem |21 differs from [6, 
Lemma 1] in two aspects. First, Theorem |21 holds for both MS and SP decoding while [6, Lemma 1] 
holds only for the SP decoding. Second, we did not keep track of the evolution of the Bhattacharryya 
parameters, which are used in [6] to bound P^^ for all I. 



4 Sum-Product Decoding Performance Analysis 



Let 



(13) 



be the log-likelihood ratio (LLR) of the input variable x given the output variable y of the MBIOS 
channel f{y\x). Moreover, let M be the random variable whose realization is m assuming x = 0, and 
g{m) be the probability density function (pdf) of M. Then the symmetry condition (Q) becomes 



g{—m) = e "^g{m), Vm G M 
Define the probability of error Pe(M) under ML decoding of the LLR M as follows 



Pe{M) 



g{m)dm. 



(14) 



(15) 



We have the following lemma. 
Lemma 3 



E 



tanh 



|M| 



2Pe(M) 



(16) 



Proof: 



E 



tanh 



|M| 



m 



tanh — ) [g{m) + g{—ni)\ dm 



\g{m) + e ™'(7(m)] dm 



/■oo 

/ (1 - e""*) g{m)dm 
Jo 



g{m)dm. 

2Pe{M) 



g{m)dm 



(17) 



Now, consider the SP decoding on a tree code used on an MBIOS channel. As shown in [2], all 
the SP decoding messages can be represented by the LLR's and satisfy the symmetry condition ()14|1 . 
Assuming that the all-zero codeword is transmitted, we have the following lemma describing the 
relationship between the probability of ML decoding errors associated with the incoming and out- 
going messages of a check node. Note that in the following, we will use capital letters to denote 
random variables, whose realizations are denoted by the corresponding lower-case letters. 

Lemma 4 Let c he a check node of degree dc- Furthermore, let M^. he the outgoing message on 
an edge, and Mi, M2, . . . , Mrf^_i the incoming messages from the other edges. Assuming all the 
incoming messages are independent with each other, we have 



dc-l 



1 - 2P,(M,) = n [1 - 2^e(M,)] 



(18) 



i=0 



Proof: It follows from Lemma El the fact that under SP decoding [2], 



tanh 



2 



tanh 



rrii 



(19) 



i=l 



and the independence of the incoming messages. ■ 
A similar relationship for a variable node is also derived. However, since the derivation is more 
complicated than the one for a check node, we first consider the simple case where the number of 
incoming messages is 2. 

Lemma 5 Let v be a variable node. Furthermore, let be the outgoing message on an edge, and 
Ml and M2 the incoming messages from either the other edges or the channel. Assuming Mi and 
M2 are the only incoming messages and independent with each other, we have 

2 

2Pe{M,) > W [2P,(M,)] (20) 



i=l 



Proof: From Lemma we have 
2 

2Pe(M„)>n[2^e(M,)] 
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(21) 



Since under SP decoding, we have from [2] that m^, = nii + m2, the left hand side of ()21|) becomes 

[1 - e-(™i+"^2)] gi{mi)g2{m2)dmidm2 



'mi+m2>0 

[1 - e-('"i+'"2)] gi{mi)g2{m2)dmidm2 

'mi>0,m2>0 

[1 - e-("^i+™2)] gi{mi)g2{m2)dmidm2 

' mi >— m2,m2 <0 

[1 - e~(™i+™2)] gi{mi)g2{m2)dmidm2 (22) 

' 7712 >— mi, mi <0 

where gi and (72 are the pdf's of Mi and M2, respectively. On the other hand, the right hand side 
becomes 



/ (1 - e-'"^) gi{mi)dmi + [ (l - e""*^) ^72(m2)rfm2 

Jmi>0 Jm2>0 

(1 - e"*"!) (1 - gi{mi)g2{m2)dmidm2 



mi>0,m2>0 

[1 - e-('"i+™2)j g^{^rni)g2{m2)dmidm2 



mi>0,m2>0 



mi >0,m2 <0 



(l-e "'^) gi{mi)g2{'m2)dmidm2+ / (l 

>/ mo>0,mi <0 



e '^^) gi{mi)g2{m2)dmidm2 

(23) 



Now, since 

/ (l — e~™') gi(mi)gj{mj)dmidmj > (l — e"'"*) gi{mi)gj{mj)dmidmj 

J mi>0,mj<0 Jmi>—mj,mj<0 

> / [l - e~'^'e~'^^) gi{mi)gj{mj)dmidmj 

J mi >—mj ,mj <0 

(24) 

for equals (1,2) and (2, 1), (j2T| is true, and the lemma is proved. ■ 
Armed with the previous lemma, we can proceed to the general case where the variable node has 
degree d^- 

Corollary 1 Let v be a variable node of degree dy. Furthermore, let Mq be the incoming message 
from the channel, My the outgoing message on an edge, and Mi, M2, . . . , Ma^_i the incoming mes- 
sages from the other edges. Assuming all the incoming messages are independent with each other, 
we have 

fill — 1 

2PeiMy) > II [2Pe(M,)] (25) 

Proof: It follows directly from Lemma El and induction. ■ 
For a tree code used on an MBIOS channel under SP decoding, all the incoming messages to the 
variable and check nodes are independent with each other. Hence, Lemma |3] and Corollary ^ can 
be used to characterize the evolution of the probability of error associated with the messages after 
processes of the variable and check nodes on a tree, and imply the following theorem. 

Theorem 3 For the (A, p) LDPC ensemble used on an MBIOS channel, the probability of bit er- 
ror Pi associated with the outgoing message of any variable node after the Ith decoding iteration 
asymptotically satisfies 

2P, >2PoA(l-p(l-2Pi„i)) (26) 

where Pq is the uncoded bit error probability under ML decoding of the channel. 

Proof: As discussed in Section the probability of error associated with the outgoing message 
of any variable node after the /th decoding iteration is, asymptotically as the codeword length goes 
to infinity, the one of the root variable node of a tree of level / + 1. Since all the variable and check 
nodes in the tree have exactly the same degree distributions A and p, respectively, and the channel 
is memoryless, all the incoming messages from child nodes to a parent node are independent and 
identically distributed. Hence, if we let c be a check node in the tree, M^ its outgoing message, and 
My one of its incoming message, then we have from Lemma 0] that 

00 

2Pe(M,) = 1 - ^ [1 - 2Pe(M,)](*-') = 1 _ p(l _ 2Pe(M,)) (27) 

i=l 

Similarly, if we let w be a variable node in the tree, Mc its outgoing message, Mq the incoming 
message from the channel, and My one of its incoming messages from its child nodes, then we have 
from Corollary ^ that 

oo 

2P,{My) > 2Pe(Mo) J2 [2Pe(M,)](*-^) A, = 2Pe(Mo)A(l - 2P,(Af,)) (28) 
1=1 



Combining ()27|) . ()28|) . and the fact that p is a monotonically increasing function (since all p^'s are 
nonnegative), the theorem is proved. ■ 
Notice that on the binary erasure channel (BEC) of erasure probability e, the probability of uncoded 
bit error of this channel is e/2. Hence, (PBj) is satisfied with equality, and recovers the well-known 
DE equation 

xi = e\ {1 - p {1 - xi-i)) , xo = e (29) 

where xi is the bit erasure probability after the Ith iteration on the BEC. Using this fact, we have 
the following corollary. 

Corollary 2 For any (A, p) LDPC ensemble, its asymptotic SP decoding performance on the MBIOS 
channel with uncoded hit error probability Pq is always worse than its asymptotic SP decoding per- 
formance on the BEC with erasure probability e < 2Pq. 

Proof: From ()26p and ()29p. we have Xi < 2Pi for all / > when e < 2Pq. Since the erasure 
probability is two times the bit error probability, the corollary is proved. ■ 
Notice that. Lemma ID and Corollary ^ can also be used in the more general family of multi- 
edge type LDPC codes [9], including the irregular repeat-accumulate (IRA) codes [10,11] and the 
low-density parity-check and generator matrix (LDPC-GM) codes [12], to produce similar results. 
Hence, Corollary El is not restricted to the irregular LDPC codes, but also holds for the general 
multi-edge type LDPC codes. 



5 Conclusion 

In this paper, we analyze the asymptotic performance of LDPC codes under MS and SP decoding 
on MBIOS channels. This is done by upper bounding the bit error probability of the root bit of the 
tree code associated with the (A, p) LDPC ensemble assuming the all-zero codeword is transmitted. 
When MS decoding is performed on this tree code, we upper bound the probability of the root bit 
being in error by the probability of sequence error under ML decoding of a subcode of the tree code. 
A recursive equation describing the evolution of the weight enumerator of this subcode after each 
iteration is then derived and used in a union bound to bound the ML decoded sequence error of this 
subcode. As a result, we obtain a recursive upper bound on the bit error probability as a function 
of the number of iterations for the LDPC codes under MS decoding on MBIOS channels. Note that 
these upper bounds are also upper bounds for the SP decoding since SP decoding is optimal on the 
bit error probability for tree codes. This result is very similar to [6, Lemma 1] with the difference 
being that we establish it not only for the SP decoding, but also for the MS decoding, and that we 
obtain it via a totally different approach. 

When SP decoding performance is considered, we derive a recursive lower bound on the proba- 
bility of bit error as a function of the number of iterations. Note that this is also a lower bound for 
the MS decoding due to the optimality of SP decoding on the bit error probability. More signifi- 
cantly, this recursion recovers the DE equation on the BEC for LDPC codes with the lower bound 
being an exact equality. This further implies that the SP decoding performance of LDPC codes 
on the BEC can serve as a upper bound of the performance on all MBIOS channels with the same 
probability of uncoded bit error. This result is also true for the more general multi-edge type LDPC 
codes, including IRA and LDPC-GM codes, since the main ingredient in the proof, i.e.. Lemma |3] 
and Corollary can also be utilized for these codes. 
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